Adolescent, parent, and provider attitudes toward a machine learning based clinical decision support system for selecting treatment for youth depression.

BACKGROUND: Machine learning based clinical decision support systems (CDSSs) have been proposed as a means of advancing personalized treatment planning for disorders, such as depression, that have a multifaceted etiology, course, and symptom profile. However, machine learning based models for treatment selection are rare in the field of psychiatry. They have also not yet been translated for use in clinical practice. Understanding key stakeholder attitudes toward machine learning based CDSSs is critical for developing plans for their implementation that promote uptake by both providers and families. METHODS: In Study 1, a prototype machine learning based Clinical Decision Support System for Youth Depression (CDSS-YD) was demonstrated to focus groups of adolescents with a diagnosis of depression (n = 9), parents (n = 11), and behavioral health providers (n = 8). Qualitative analysis was used to assess their attitudes towards the CDSS-YD. In Study 2, behavioral health providers were trained in the use of the CDSS-YD and they utilized the CDSS-YD in a clinical encounter with 6 adolescents and their parents as part of their treatment planning discussion. Following the appointment, providers, parents, and adolescents completed a survey about their attitudes regarding the use of the CDSS-YD. RESULTS: All stakeholder groups viewed the CDSS-YD as an easy to understand and useful tool for making personalized treatment decisions, and families and providers were able to successfully use the CDSS-YD in clinical encounters. Parents and adolescents viewed their providers as having a critical role in the use the CDSS-YD, and this had implications for the perceived trustworthiness of the CDSS-YD. Providers reported that clinic productivity metrics would be the primary barrier to CDSS-YD implementation, with the creation of protected time for training, preparation, and use as a key facilitator. CONCLUSIONS: Machine learning based CDSSs, if proven effective, have the potential to be widely accepted tools for personalized treatment planning. Successful implementation will require addressing the system-level barrier of having sufficient time and energy to integrate it into practice.

Adolescent, parent, and provider attitudes toward a machine-learning based clinical decision support system for selecting treatment for youth depression.

Background: Machine-learning based clinical decision support systems (CDSSs) have been proposed as a means of advancing personalized treatment planning for disorders, such as depression, that have a multifaceted etiology, course, and symptom profile. However, machine-learning based models for treatment selection are rare in the field of psychiatry. They have also not yet been translated for use in clinical practice. Understanding key stakeholder attitudes toward machine learning-based CDSSs is critical for developing plans for their implementation that promote uptake by both providers and families. Methods: In Study 1, a machine-learning based Clinical Decision Support System for Youth Depression (CDSS-YD) was demonstrated to focus groups of adolescents with a diagnosis of depression (n = 9), parents (n = 11), and behavioral health providers (n = 8). Qualitative analysis was used to assess their attitudes towards the CDSS-YD. In Study 2, behavioral health providers were trained in the use of the CDSS-YD and they utilized the CDSS-YD in a clinical encounter with 6 adolescents and their parents as part of their treatment planning discussion. Following the appointment, providers, parents, and adolescents completed a survey about their attitudes regarding the use of the CDSS-YD. Results: All stakeholder groups viewed the CDSS-YD as an easy to understand and useful tool for making personalized treatment decisions, and families and providers were able to successfully use the CDSS-YD in clinical encounters. Parents and adolescents viewed their providers as having a critical role in the use the CDSS-YD, and this had implications for the perceived trustworthiness of the CDSS-YD. Providers reported that clinic productivity metrics would be the primary barrier to CDSS-YD implementation, with the creation of protected time for training, preparation, and use as a key facilitator. Conclusions: The CDSS-YD has the potential to be a widely accepted and useful tool for personalized treatment planning. Successful implementation will require addressing the system-level barrier of having sufficient time and energy to integrate it into practice.

Toward a more comprehensive understanding of organizational influences on implementation: the organization theory for implementation science framework.

Introduction: Implementation is influenced by factors beyond individual clinical settings. Nevertheless, implementation research often focuses on factors related to individual providers and practices, potentially due to limitations of available frameworks. Extant frameworks do not adequately capture the myriad organizational influences on implementation. Organization theories capture diverse organizational influences but remain underused in implementation science. To advance their use among implementation scientists, we distilled 70 constructs from nine organization theories identified in our previous work into theoretical domains in the Organization Theory for Implementation Science (OTIS) framework. Methods: The process of distilling organization theory constructs into domains involved concept mapping and iterative consensus-building. First, we recruited organization and implementation scientists to participate in an online concept mapping exercise in which they sorted organization theory constructs into domains representing similar theoretical concepts. Multidimensional scaling and hierarchical cluster analyses were used to produce visual representations (clusters) of the relationships among constructs in concept maps. Second, to interpret concept maps, we engaged members of the Cancer Prevention and Control Research Network (CPCRN) OTIS workgroup in consensus-building discussions. Results: Twenty-four experts participated in concept mapping. Based on resulting construct groupings' coherence, OTIS workgroup members selected the 10-cluster solution (from options of 7-13 clusters) and then reorganized clusters in consensus-building discussions to increase coherence. This process yielded six final OTIS domains: organizational characteristics (e.g., size; age); governance and operations (e.g., organizational and social subsystems); tasks and processes (e.g., technology cycles; excess capacity); knowledge and learning (e.g., tacit knowledge; sense making); characteristics of a population of organizations (e.g., isomorphism; selection pressure); and interorganizational relationships (e.g., dominance; interdependence). Discussion: Organizational influences on implementation are poorly understood, in part due to the limitations of extant frameworks. To improve understanding of organizational influences on implementation, we distilled 70 constructs from nine organization theories into six domains. Applications of the OTIS framework will enhance understanding of organizational influences on implementation, promote theory-driven strategies for organizational change, improve understanding of mechanisms underlying relationships between OTIS constructs and implementation, and allow for framework refinement. Next steps include testing the OTIS framework in implementation research and adapting it for use among policymakers and practitioners.

Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy.

BACKGROUND: Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers' implementation of evidence-based psychosocial interventions-(1) audit and feedback and (2) active learning. METHODS: Items were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples. RESULTS: In sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ2=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ2=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples. CONCLUSIONS: Scores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings. TRIAL REGISTRATION: ClinicalTrials.gov NCT04096274 . Registered on 19 September 2019.

In Search of the Common Elements of Clinical Supervision: A Systematic Review.

The importance of clinical supervision for supporting effective implementation of evidence-based treatments (EBTs) is widely accepted; however, very little is known about which supervision practice elements contribute to implementation effectiveness. This systematic review aimed to generate a taxonomy of empirically-supported supervision practice elements that have been used in treatment trials and shown to independently predict improved EBT implementation. Supervision practice elements were identified using a two-phase, empirically-validated distillation process. In Phase I, a systematic review identified supervision protocols that had evidence of effectiveness based on (a) inclusion in one or more EBT trials, and (b) independent association with improved EBT implementation in one or more secondary studies. In Phase II, a hybrid deductive-inductive coding process was applied to the supervision protocols to characterize the nature and frequency of supervision practice elements across EBTs. Twenty-one of the 876 identified articles assessed the associations of supervision protocols with implementation or clinical outcomes, representing 13 separate studies. Coding and distillation of the supervision protocols resulted in a taxonomy of 21 supervision practice elements. The most frequently used elements were: reviewing supervisees' practice (92%; n = 12), clinical suggestions (85%; n = 11), behavioral rehearsal (77%; n = 10), elicitation (77%; n = 10), and fidelity assessment (77%; n = 10). This review identified supervision practice elements that could be targets for future research testing which elements are necessary and sufficient to support effective EBT implementation. Discrepancies between supervision practice elements observed in trials as compared to routine practice highlights the importance of research addressing supervision-focused implementation strategies.

Increasing Access to Organization Theories for Implementation Science.

Background: Organization theories offer numerous existing, highly relevant, yet largely untapped explanations of the organizational dynamics underlying evidence-based intervention (EBI) implementation. Rooted in ideas regarding power, autonomy, and control, organization theories can explain how and why organizations adopt, implement, and sustain EBI use. Although they have gained visibility, organization theories remain underused in implementation research, perhaps due to their inaccessibility to implementation scientists. To improve access to organization theory among implementation scientists, we summarized organization theories with relevance to implementation science. Methods: Led by the Cancer Prevention and Control Research Network (CPCRN) Organization Theory for Implementation Science workgroup, we employed a modified Delphi process to reach a consensus among 18 experts at the intersection of organization and implementation science regarding organization theories with relevance to implementation science. From texts that described the organization theories, using standardized abstraction forms, two investigators independently abstracted information regarding constructs, propositions regarding how or why constructs might influence implementation, the potential relevance of organization theories' propositions for implementation, and overviews of each theory. The investigators then reconciled discrepancies until reaching consensus. A third investigator reviewed reconciled abstraction forms for accuracy, coherence, and completeness. Findings: We identified nine organization theories with relevance to implementation science: contingency, complexity, institutional, network, organizational learning, resource dependence, sociotechnical, and transaction cost economics. From the theories, we abstracted 70 constructs and 65 propositions. An example proposition from institutional theory is: "Coercive, mimetic, and normative pressures contribute to organizations…within an organizational field [becoming increasingly similar]." These propositions can be operationalized as levers to facilitate EBI implementation. Conclusions: To increase use in the field, organization theories must be made more accessible to implementation scientists. The abstraction forms developed in this study are now publicly available on the CPCRN website with the goal of increasing access to organization theories among an interdisciplinary audience of implementation scientists through the CPCRN Scholars program and other venues. Next steps include consolidating organization theory constructs into domains and translating the resulting framework for use among researchers, policymakers and practitioners, aiding them in accounting for a comprehensive set of organization theory constructs thought to influence EBI implementation.

Perception of Recovery Orientation as a Predictor of Clinician Turnover in Community Mental Health Clinics.

Turnover in community mental health clinics threaten the delivery of quality behavioral health services; recovery orientation has been shown to be associated with lower levels of burnout but its relationship with actual turnover has not yet been examined. This study aimed to examine the relationship between provider perceptions of recovery orientation and 12-month turnover status among community mental health providers. Secondary data analyses were conducted with data collected from 224 community mental health providers from ten organizations across 14 clinic sites participating in a larger effectiveness-implementation trial. Chi square analyses were used to examine between-site variation in 1-year turnover rates among the ten organizations. Logistic regression was used to examine effects of perceptions of recovery orientation along with known predictors of turnover among community mental health providers. Results revealed no between-site variation in organizational turnover rates. The logistic regression model was statistically significant, χ2(17) = 47.64, p < .0001 and explaining 30% (Nagelkerke R2) of within-site variance and correctly classifying 79% of cases. Perceptions of recovery orientation emerged as the sole significant predictor of 12-month turnover status, with higher levels of recovery orientation significantly associated with lower odds of turnover at 12 months [B = .90, p = .008, OR 2.47 (95% CI 1.23, 4.82)]. Perceptions of recovery orientation was protective against 12-month turnover status suggesting that a more person-centered and empowering approach to care may improve the provider experience. More research is needed to further understand how recovery orientation can contribute to workforce stability.

Implementing Person-Centered Care Planning: A Randomized Controlled Trial.

OBJECTIVE: Person-centered care is a key quality indicator and central to promoting integrated and recovery-oriented services. Person-centered care planning (PCCP) is a manualized intervention promoting the collaborative cocreation of a recovery-oriented care service plan on the basis of an individual's most valued life goals. This cluster randomized controlled trial tested the effect of PCCP training on person-centered care delivery in community mental health clinics. METHODS: Fourteen clinic sites were randomly assigned to receive either PCCP training (N=7; experimental condition) or service planning as usual (N=7; control condition). Data were collected from online surveys, and service plans were completed by 60 provider teams. The Person-Centered Care Planning Assessment Measure was administered via chart review at baseline, 12 months, and 18 months, and surveys were used to measure supervision, implementation leadership, and program type. The main effect was examined with linear mixed-effects regression models, with observations over time. RESULTS: Analyses controlling for service user and program characteristics revealed that at 12 months, the group assigned to PCCP training showed significant improvements in delivering person-centered care compared with the control group (b=1.10, SE=0.50, p=0.03). At 18 months, this effect was even more pronounced (b=1.47, SE=0.50, p=0.01), representing a medium-to-large effect size of d=0.71 (95% confidence interval=0.23-1.20). CONCLUSIONS: These findings indicate that training providers in PCCP increases provider competency in delivering person-centered care. Using an objective measure of person-centered care, the authors show that a comprehensive training strategy can target both the philosophical shift and the technical skills needed to promote client recovery.

"I see your punitive measure and I raise you a person-centered bar": Supervisory Strategies to Promote Adoption of Person-Centered Care.

Person-centered care remains a high priority within community mental health services. Clinical supervision is an embedded resource for professional development and promotion of high quality care. This study examined supervisory strategies during the implementation of person-centered care planning (PCCP) across two northeastern US States. A criterion sample of supervisor-provider teams participated in qualitative interviews (N = 34) and direct observation from 2016 to 2017. Modified grounded theory analyses were conducted and three supervisory strategies were identified. Supervisory attunement to providers (knowing their audience), active collaborative engagement with providers (practicing together), and infusing reminders and opportunities for feedback (chipping away) were critical strategies to engage providers in adopting PCCP. These strategies changed providers' practice patterns by improving supervisors' calibration to dynamic contextual and individual needs during implementation and communicating supervisors' expectations of PCCP enactment. Workplace-based clinical supervision holds promise as a key intervention point to embed high quality care.

Strategies for Sustaining Fidelity: A Multi-state Qualitative Analysis in Housing First Programs.

Little is known about long-term fidelity of evidence-based interventions (EBIs) under changing conditions. This study examines how staff at 'mature' (eight or more years in operation) Housing First (HF) programs strategize to sustain EBI fit in different geographic areas in the Mid-Atlantic/Northeastern United States. Six focus groups (FGs) at three purposively selected HF programs were conducted with separate FGs for case managers and supervisors at each site. FG discussions elicited participants' service approaches and strategies in addressing fidelity amidst ongoing changes affecting each program. Thematic content analysis of FG transcripts was conducted using the five HF fidelity domains (housing choice/structure, separation of housing and services, service philosophy, service array, and program structure) as a priori themes with inductive content analyses conducted on data in each theme. Strategies for rigor were employed. Case managers (N = 17) and supervisors (N = 16) were predominantly white (76%) and female (60%). Across the themes, challenges included lack of affordable housing and choice, funders' restrictions and practice 'drift.' Strategies included community engagement and hiring, strong leadership and 'bending the rules.' There were no differences across sites. Later-stage implementation challenges show the need for continued vigilance in fidelity to EBIs. Among the strategies used to address fidelity in this study, the pursuit of pro-active community engagement to attract knowledgeable staff as well as increase local buy-in was considered pivotal at all three sites. These findings underscore the need to attend to the external setting as well as to internal program operations.

Do Social Workers Lead Differently? Examining Associations with Leadership Style and Organizational Factors.

What training leaders need to successfully shepherd high quality human service delivery remains less understood. This study aims to evaluate associations between social work training, leadership style, and organizational factors. Participants included leaders (N=49) and employees (N=224) working in 10 behavioral health organizations who completed a survey. Hierarchical linear models were used to analyze multilevel associations. Analyses indicated that leaders with social work training were rated higher in transformational leadership and organizational factors (e.g., lesser stress) on average within both samples. Findings begin an empirical argument for the value of social work leaders for the success of human service organizations.

Getting to the table: Agency characteristics and evidence-based intervention adoption in children's mental health care.

Scaling evidence-based interventions (EBI) for children and families across healthcare systems can expand public health impact. Research has identified EBI adoption determinants. However, less understood are characteristics of agencies that opt in across the stages of adoption. This study examined the relationship between agency (N=69) characteristics (e.g., revenue) and four adoption stages during a large-scale trial of an EBI for children with significant behavioral difficulties and their families. 48 (70%) of agencies demonstrated interest, 28 (41%) scheduled an informational meeting, 20 (29%) received training, and 16 (22%) demonstrated EBI uptake. Analyses indicated no differences in characteristics and initial interest. However, agencies with small-sized revenue had significantly reduced odds at other adoption stages. Implications for strategies to bring EBI access to scale are discussed.

Elucidating the influence of supervisors' roles on implementation climate.

BACKGROUND: Supervisors play an essential role in implementation by diffusing and synthesizing information, selling implementation, and translating top management's project plans to frontline workers. Theory and emerging evidence suggest that through these roles, supervisors shape implementation climate-i.e., the degree to which innovations are expected, supported, and rewarded. However, it is unclear exactly how supervisors carry out each of these roles in ways that contribute to implementation climate-this represents a gap in the understanding of the causal mechanisms that link supervisors' behavior with implementation climate. This study examined how supervisors' performance of each of these roles influences three core implementation climate domains (expectations, supports, and rewards). MATERIALS AND METHODS: A sequenced behavioral health screening, assessment, and referral intervention was implemented within a county-based child welfare agency. We conducted 6 focus groups with supervisors and frontline workers from implementing work units 6 months post-implementation (n = 51) and 1 year later (n = 40) (12 groups total). Participants were asked about implementation determinants, including supervision and implementation context. We audio-recorded, transcribed, and analyzed focus groups using an open coding process during which the importance of the supervisors' roles emerged as a major theme. We further analyzed this code using concepts and definitions related to middle managers' roles and implementation climate. RESULTS: In this work setting, supervisors (1) diffused information about the intervention proactively, and in response to workers' questions, (2) synthesized information by tailoring it to workers' individual needs, (3) translated top managements' project plans into day-to-day tasks through close monitoring and reminders, and (4) justified implementation. All four of these roles appeared to shape the implementation climate by conveying strong expectations for implementation. Three roles (diffusing, synthesizing, and mediating) influenced climate by supporting workers during implementation. Only one role (diffusing) influenced climate by conveying rewards. CONCLUSIONS: Supervisors shaped implementation climate by carrying out four roles (diffusing, synthesizing, mediating, and selling). Findings suggest that the interaction of these roles convey expectations and support for implementation (two implementation climate domains). Our study advances the causal theory explaining how supervisors' behavior shapes the implementation climate, which can inform implementation practice.

Advancing the use of organization theory in implementation science.

Healthcare settings and systems have been slow to adopt and implement many effective cancer prevention and control interventions. Understanding the factors that determine successful implementation is essential to accelerating the translation of effective interventions into practice. Many scholars have studied the determinants of implementation, and much of this research has been guided by the Consolidated Framework for Implementation Research (CFIR). The CFIR categorizes implementation determinants at five levels (characteristics of the intervention, inner setting, individual, processes, and outer setting). Of these five levels, determinants at the level of the outer setting are the least developed. Extensive research in fields other than healthcare suggest that determinants at the level of the outer setting (e.g., funding streams, contracting practices, and public policy) play a central role in shaping when and how an organization implements new structures and practices. Thus, a more comprehensive understanding of outer-setting determinants is critical to efforts to accelerate the implementation of effective cancer control interventions. The Cancer Prevention and Control Research Network (CPCRN) created a cross-center workgroup to review organizational theories and begin to contribute to the creation of a future framework of constructs related to outer setting determinants. In this paper, we report findings from the review of three organizational theories: Institutional Theory, Transaction Cost Economics, and Contingency Theory. To demonstrate the applicability of this work to implementation science and practice, we have applied findings to three case studies of CPCRN researchers' efforts to implement colorectal cancer screening interventions in Federally Qualified Health Centers.

A Mixed Methods Study of Organizational Readiness for Change and Leadership During a Training Initiative Within Community Mental Health Clinics.

This longitudinal mixed-methods study explored variation in organizational readiness for change and leadership behavior across seven organizations during a 12-month training initiative in person-centered care planning. Quantitative data was used to examine trajectories of organizational readiness for change and leadership behavior over time and qualitative data explored provider perspectives on the trajectory of these organizational factors during the 12-month training initiative. Findings indicated that levels of organizational readiness for change and leadership behavior varied across clinics, but most experienced a significant positive change at the mid-point of the training. Organizational readiness for change was positively correlated with leaderships behaviors across time. Provider focus group findings gave insight into their initial resistance to adopting the new practice and their increasing receptivity in the second 6 months due to increased understanding of the practice and leadership endorsement. Increasing provider openness to a new practice prior to training and having a consistently engaged leadership have the potential to improve the efficiency of a training initiative.

Person-Centered Care and the Therapeutic Alliance.

Person-centered care has been gaining prominence in behavioral health care, and service planning has shifted towards "person-centered care planning" (PCCP), where individuals, in partnership with providers, identify life goals and interventions. A strong therapeutic alliance has been identified as key to a person-centered approach, but little is known about how the therapeutic relationship influences person-centered processes and outcomes. Using an explanatory sequential mixed methods design, this study investigated: (1) the association between the therapeutic alliance and PCCP, and (2) how the therapeutic relationship influences the process and outcomes of PCCP. Quantitative analyses found that a strong working alliance predicted greater personcenteredness. Qualitative analyses revealed two central themes: (1) the importance of connection, continuity, and calibration of the relationship to set the right conditions for PCCP, and (2) PCCP as a vehicle for engagement. Findings demonstrated that the therapeutic alliance is inextricably linked to the PCCP process, each influencing the other.

Do Providers Know What They Do Not Know? A Correlational Study of Knowledge Acquisition and Person-Centered Care.

Person-centered care (PCC) is a central feature of health care reform, yet the tools needed to deliver this practice have not been implemented consistently. Person-centered care planning (PCCP) is a treatment planning approach operationalizing the values of recovery. To better understand PCCP implementation, this study examined the relationship between recovery knowledge and self-reported PCCP behaviors among 224 community mental health center staff. Results indicated that increased knowledge decreased the likelihood of endorsing non-recovery implementation barriers and self-reporting a high level of PCCP implementation. Findings suggest that individuals have difficulty assessing their performance, and point to the importance of objective fidelity measures.

The Availability of Supervision in Routine Mental Health Care.

Clinical supervision is an embedded resource for practice quality in community mental health organizations. Supervision has been found to increase provider competence and decrease stress. In addition, supervision has been associated with service user outcomes including decreased depressive symptoms. However, little is known about the availability and nature of supervision in real world settings. The primary aims of this study were to identify available supervision and the extent to which contextual factors are related to that availability. The data source for this study was a multi-state and multi-site (N=14) NIMH-funded trial survey of providers (N=273). Supervision was measured by hours per week (quantity) and by utilization of best practice activities (content). Univariate, chi-square, independent samples t-tests, and ANOVA analyses were used to assess supervision content and quantity and to examine subgroup differences. Participants reported an average of 2.17 hours of supervision per week and 28.6% of participants endorsed best practice content. Supervision quantity varied significantly across sites (p<.05) and program type (p<.05) while content did not. Individual role within the organization had a significant relationship with reported supervision content (p<.001). In these settings, organizations are exercising discretion in how to utilize supervision within the available time. Supervision time also varied by program type, increasing with the intensity of services. Findings demonstrate that reports of availability vary according to position within the organization and the intensity of services within a given program type. Implications for workforce development, access to quality services, and implementation of evidence-based practices are discussed.

A comparison of how behavioral health organizations utilize training to prepare for health care reform.

BACKGROUND: Under the Affordable Care Act, States have obtained Medicaid waivers to overhaul their behavioral health service systems to improve quality and reduce costs. Critical to implementation of broad service delivery reforms has been the preparation of organizations responsible for service delivery. This study focused on one large-scale initiative to overhaul its service system with the goal of improving service quality and reducing costs. The study examined the participation of behavioral health organizations in technical assistance efforts and the extent to which organizational factors related to their participation. METHODS: This study matched two datasets to examine the organizational characteristics and training participation for 196 behavioral health organizations. Organizational characteristics were drawn from the Substance Abuse and Mental Health Services Administration National Mental Health Services Survey (N-MHSS). Training variables were drawn from the Clinical Technical Assistance Center's master training database. Chi-square analyses and multivariate logistic regression models were used to examine the proportion of organizations that participated in training, the organizational characteristics (size, population served, service quality, infrastructure) that predicted participation in training, and for those who participated, the type (clinical or business) and intensity of training (webinar, learning collaborative, in-person) they received. RESULTS: Overall 142 (72. 4%) of the sample participated in training. Organizations who pursued training were more likely to be large in size (p = .02), serve children in addition to adults (p < .01), provide child evidence-based practices (p = .01), and use computerized scheduling (p = .01). Of those trained, 95% participated in webinars, 64% participated in learning collaboratives and 35% participated in in-person trainings. More organizations participated in business trainings than clinical (63.8 vs. 59.2%). Organizations serving children had higher odds of participating in both clinical training (OR = 5.91, p < .01) and business training (OR = 4.24, p < .01) than those that did not serve children. CONCLUSIONS: The majority of organizations participated in trainings indicating desire for technical assistance to prepare for health care reform. Larger organizations and organizations serving children were more likely to participate potentially indicating increased interest in preparation. Over half participated in business trainings highlighting interest in learning to improve efficiency. Further understanding is needed to support organizational readiness for health care reform initiatives among behavioral health organizations.